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1 Introduction 

Equirecursive types consider a recursive type to be equal to its unrolling and have 
no explicit term-level coercions to change a term's type from the former to the 
latter or vice versa. This equality makes deciding type equality and subtyping 
more difficult than the other approach — isorecursive types, in which the types 
are not equal, but isomorphic, witnessed by explicit term-level coercions. Pre- 
vious work has built intuition, rules, and polynomial-time decision procedures 
for equirecursive types for first-order type systems. Some work has been done 
for type systems with parametric polymorphism, but that work is incomplete 
(see below). This chapter will give an intuitive theory of equirecursive types for 
second-order type systems, sound and complete rules, and a decision procedure 
for subtyping. 

Another interesting feature of type systems turns out to be quite related to 
equirecursive types. Canning et al. [CCH+89] introduced the idea of F-bounded 
polymorphism. In this form of polymorphism a type bound can mention the 
type being bounded. For example, it can require a type that has a method 
that returns an object of the type being bounded. This form of bound is useful 
for binary methods and in typing object encodings [GleOO , . Treating a type 
variable as being a subtype of its bound when that bound can refer to it is 
like treating a recursive type as being equal to its unrolling, and similar issues 
arise to formalising such type systems. This chapter will also treat F-bounded 
parameteric polymorphism, and give it an intuitive formalisation, sound and 
complete set of rules, and decision procedure for subtyping. 

Amadio and Cardelli |AC93j were the first to investigate the equirecursive 
approach. They proposed the tree interpretation of recursive types, which is 
based on the idea of repeatedly unrolling recursive types into possibly-infinite 
trees. Two types are equal if their corresponding trees are the same; similarly, 
subtyping can be defined on trees and lifted to types. Amadio and Cardelli made 
these ideas precise, defined a set of rules for type equality and subtyping that 
are sound and complete, and provided an exponential-time decision procedure 
for equality and subtyping. Kozen et al. [KPS95] reduced this exponential time 
to quadratic time, by defining a notion of tree automata that generate trees just 
as types do and a construction from two tree automata that decides equality 
and subtyping. Both Amadio and Cardelli and Kozen et al. worked with first- 
order type systems. Colazzo and Ghelli |CC99j investigated a second-order type 



system with equirecursion. They gave a coinductive set of rules for subtyping and 
a decision procedure, but did not relate their rules to trees nor show soundness 
and completeness to any intuitive model. In previous work Glc02a Gle02b], I 
gave a tree model for second-order systems, a set of rules for equality of types, 
showed soundness and completeness of those rules, defined automata for the 
trees, and defined a construction on automata that gives a decision procedure 
for equality^ However, I did not address subtyping. Gauthier and Pottier [GP04] 
devised an O(nlogn) algorithm to decide equality of second-order types with 
equirecursive types by reducing second-order types to canonical first-order types 
in a particular way such that the canonical types are equal exactly when the 
original types are equal; their approach also handles entailment of type equations 
and type unification problems with the same complexity. 

This chapter investigates a second-order type system with F-bounded quan- 
tifiers and equirecursive types. First, it defines a notion of trees that provide an 
intuitive model for such types and defines an intuitive notion of subtyping on 
these trees. Next, it presents the types themselves and defines how these map 
to trees. Then it presents a set of type equality and subtyping rules and shows 
soundness and completeness of these rules with respect to the tree interpreta- 
tion. Finally, it defines a notion of tree automata, how these generate trees, and 
a construction that decides subtyping in polynomial time. 

Complete definitions and proofs of all the results in this chapter appear in a 
companion technical report |Glel2) . 

2 Binding Trees 

I consider a system with top, bottom, function, and F-bounded forall quantified 
types. To model such types, I use possibly-infinite trees over these constructs 
and de Bruijn indices to model type variables. These trees are formulated in the 
standard way: 

Tree = {t : {L, R}* N U {T, _L, V} | 

e € dom(t) A {pi G dom(t) ^> t{p) E {^,V})} 

Forall quantifiers are F-bounded, that means that V binds a variable in each of 
its subtrees — in the left subtree, the bound, to refer to the type being bounded 
and in the right tree to refer to the quantified type. 

Trees form a complete 1-bounded ultrametic space in the usual way. Some 
sample trees appear in Figure [TJ and I overload the notation and use var(n), T, 
_!_, t\_ — >tRi and V£|_.^r to denote appropriate trees. The subtree of t along path p 
is subtree(t,p), the number of variables bound along the path from pi to p^ (a 
suffix) is bind(t,pi — > p 2 )> and shifting the free variables of a tree t up by n is 
shift(t,n); their definitions are straightforward. 

1 In those papers I claimed that the algorithm could be made quadratic time. I have 
recently realised that the reasoning was incorrect, and thus that my papers demon- 
strated only an exponential-time algorithm. The arguments I give latter can be used 
to make an 0(n 4 ) algorithm from the ones in my previous work. 




Fig. 1. Examples trees tl, on left, and t2, on right. 



2.1 Regular Trees 



Not all trees are generated by syntactic types. For first-order systems, regular 
trees — those with a finite number of subtrees — correspond exactly to the trees 
that can be generated. For the trees defined above, this is no longer the case. 
Consider tree tl in Figure Q] that is generated by Va.rec f3.a — > \/(3'.a — all the 
de Bruijn indices intuitively represent the same thing, the variable bound by the 
V at the top of the tree, but the actual numbers are all different. Now consider 
tree t2 in Figure [TJ It has only a finite number of subtrees, but in fact no type 
can generate it. It looks like the second and subsequent forall trees repeat each 
other, however the de Bruijn indices 1 in that tree refer to the quantifier in the 
previous iteration of the cycle rather than the current iteration, and types cannot 
generate such a structure. My previous work defined a rather complicated notion 
of regular binding trees. Since then I have discovered a better, more intuitive 
formulation. 

The idea is that relevant subtrees are equal modulo appropriate changes to 
the de Bruijn indices. For example, in the tree for rec a.V/3 < T.a, the tree itself 
is not equal to the right subtree, but the tree itself shifted by one is equal to the 
right subtree — one being the number of variables bound from the root to the 
right subtree. Generalising a bit, two subtrees at paths p\ and p 2 with greatest 
common prefix p that are intuitively equal cannot refer to any of the variables 
bound along the paths p to p\ or p 2 and will be equal modulo adjusting the 
de Bruijn indices according to the difference in the number of variables bound 
along these paths. 

This idea can be formalised by defining an equivalence relation on the paths 
of a tree t, eqst(t), that intuitively says which subtrees of t are equal. First, 
forbid(t, m) captures the notion that t does not refer to the first m free variables 
and shifts t downwards by m. It is defined when there is no p G dom(i) such that 
t(p) = n and bind(t, e — >• p) < n < bind(t, e — > p) + m, otherwise it is undefined. 



When it is denned: 



. , v . f n — m tip) = n A n > bindit, e — y p) 
foroid(t,m) = Xp. < ,, , V 

v ' \t(p) otherwise 

Now, pi eqst(t) P2 exactly when there exists p, Pi, p' 2 , and q such that all of: 

Pi = PPiQ 
P2 = pp' 2 q 

forbid(subtree(t, pp[) , bind(t,p — > pp[)) — 
forbid (subtree (t,pp' 2 ), bind(t,p — > pp' 2 )) 

(The last one when both are defined and equal.) In other words, two paths 
represent equal subtrees of t when they are common paths in two subtrees of t 
that are equal when their de Bruijn indices are adjusted to the common prefix 
of the paths to those subtrees. 

It is easy to prove that eqst(t) is an equivalence relation on dom(t). A tree t 
is a regular binding tree exactly when [eqst(t)\ is finite. 

Each equivalence classes of eqst(t) has a particular type constructor, as de- 
fined by the function: 



fv(n) 


t(p) = n + bind(t, e — > p) 


bv([p'] 


e g st(t)J) p'£<pAt(p' 1 )=VAt(p) 


T 


t(p) = T 




t(p) = _L 


— > 


tip) = -+ 


V 

V 


tip) - v 



NL (t, \p}eqst{t)) 



It is easy to prove that this function is well defined and that [p\ egs t(t)£ = [pP\eqst(t) 
when NL(t, \p} eqst (t)) G {— ^V} is also well defined. 



2.2 Subtyping 

Intuitively, subtyping of trees should be as follows. Top should be a supertype 
of everything, bottom should be a subtype of everything, a variable should be a 
subtype of itself and its bound, a function tree is a subtype of another function 
tree when the argument trees are related contravariantly and the result trees 
are related covariantly, and similarly for forall trees. This could be made into a 
formal definition by using coinduction except for a couple of points. First, if the 
coinductive definition includes the condition that a variable is a subtype of any 
tree its bound is a subtype of then problems result. In particular, if free variable 
is bounded by itself then under this definition free variable is a subtype of any 
tree because of the coinduction — what we really want is induction for bounds. 
Second, comparing ViiL-^iR to \/t 2 i-t 2 R requires selecting a bound for the variable 
bound by the Vs to do the comparison of in_ against t 2 \_ and iiR against t 2 R. The 
most general rule that is sound uses the tightest bound for the variable — this is 
t 2 i. However, that rule leads to an undecidable subtyping relation |Pie94j . The 



system considered here is actually a non-conversative extension^ but I believe 
that the undecidability still holds (but have not proven this yet). Therefore, to 
regain decidability, I will use the Kernel rule for forall — the bound is required 
to be invariant (tn — t 2 \_) and then the bound to use is the equal bound. 

To formalise the above intuition, there are several pieces to build up, as 
mixing induction and coinduction is a little tricky. A bound set, ranged over by 
metavariable /3, is a function from de Bruijn indices to trees, BSet = N — > Tree. 
Shifting the free variables up and adding a bound t for the new free variable 
is shift(/3,t), and is straightforward to define. Tree promotion, a relation <—} on 
Tree x BSet x Tree, is defined as var(n) f3(n) (no other trees are related by 
^(j)- The base subtyping proposition bst(t\, R, /3, t 2 ) holds exactly when either: 

— tx = var(n) and t 2 = var(n), 

— h = T, 

— h = ±, 

— ti(e) = — >, t 2 {e) = - subtree(t 2 , L) Rp subtree(ti, L), and subtree(t\, R) Rp 
subtree(t 2 , R), or 

— fi(e) = V, t 2 (e) — V, subtree (t 1, L) = subtreefa, L), and: 

subtree(ti, R) R s hift(j3, S ubtree(t 2 x)) subtree{t 2 , R) 

A three place relation R on Tree x BSet x Tree is a partial subtyping exactly 
when t\ R/3 ti implies that there exists t[ such that t\ t\ and bst(t[,R, /3, ^2)- 
Subtyping for trees, <, is the union of all partial subtypings. 

Subtyping satisfies several important properties justifying that the formal 
definitions do capture the right intuition. 

Theorem 1. Subtyping is a partial subtyping; <p is a preorder on Tree for any 
13 G BSet; t\ <p t 2 if and only if one of the following holds: 

— t\ = var(n) and t 2 = var(n), 

— t\ = var(n) and /3(n) <p t 2 , 
-t 2 = T, 

-h = ±, 

— tx = hi->ti 2 , t 2 = t 2 \^t 22 , t 2 i <p tu, and ti 2 <p t 22 , or 

— ti — Vt 3 .t4, t 2 = Vt 3 .t 5 , and t<i < s hift(i3,t 3 ) ^5- 

2.3 Characterising Subtyping 

A (tree) subtyping problem is a triple (i|_,/3,^R) where t\_ <p t^ might or might 
not hold. The definition of subtyping says that after promoting the subtype to 
its bound some finite number of times the two trees have to match in a certain 
sense. In particular, two trees match when they are the same de Bruijn index, 

2 The system considered by Pierce does not include recursive types of either flavour, 
but the rules in this paper could be used to define a more permissive subtyping 
relation that is still sound for Pierce's system — in essence certain subtyping that has 
an infinite derivation in Pierce's rules would be allowed rather than rejected. 



the supertype is top, the subtype is bottom, both trees are functions, or both 
trees are forall quantifiers and their respective left subtrees are equal. With this 
definition, subtyping requires that after a finite number of promotions of the 
subtype the two trees must match and furthermore, if they match because they 
are functions then the respective left subtrees must be contravariantly related 
and the respective right subtrees must be covariantly related, and if they match 
because they are forall quantifiers then the respective right subtrees must be 
covariantly related. For these various subtrees we can repeat this process, finding 
matching trees for them, and so on. Thus for some set of paths we get trees that 
match if the original subtyping held. 

We can formalise this idea as follows. A subproblem of stp will be a triple 
(t\_,/3,tR) where t\_ is the current subtype, tR is the current supertype, and j3 is 
the current bound set. For a subtyping problem sip, the initial subproblem for 
path e is simply stp. If the initial subproblem for path p is (t\_,/3,tR) and there 
is a t' L such that t\_ t' L and t' L and tR match then the final subproblem for 
path p is (t' L , (3, £r); otherwise the final subproblem for path p fails. If the final 
subproblem for path p is (ti,f3,tf>) and matches because both trees are func- 
tions then the initial subproblem for path pL is (subtree(tR, L),/3, subtree(t\_, L)) 
and the initial subproblem for path pR is (subtree(t\_, R), j3, subtree(tR 7 R)). Sim- 
ilarly, if both trees are forall quantifiers then the initial subproblem for path 
pR is (subtree(ti, R), shift((3, subtree(t\_, L)), subtree(tR, R)). Thus any subtyping 
problem has a prefixed closed set of paths containing e of initial and final sub- 
problems. 

The characterisation of subtyping is the following theorem. 

Theorem 2. t\_ <p tR if and only if all final subproblems of (ti_,(3,tR) do not 
fail. 

We can go further than just these definitions however. Each tree that appears 
in a subproblem comes, in some sense, from either of the original trees or one of 
the bounds. De Bruijn indices that are bounded by themselves (a trivial bound) 
are not interesting, so a tree identifier for a subtyping problem (£|_,/3,£r) is 
cither L, R, or n where /3(n) ^ var(n). A node identifier is a tree identifier and 
a sequence of L, R, and Ss specifying to take the left subtree, right subtree, or 
shift by one starting from that tree. Any subproblem of stp can be represented 
as a triple (nil, nis, «ir) where ni\_ is a node identifier representing the subtype, 
niR is a node identifiers representing the supertype, and nis is a sequence of 
node identifiers representing the bounds of the binding variables that have been 
opened up. We can inductively define two partial maps that compute the node 
identifier representations of the initial and final subproblems or F for a final 
subproblem that fails. 

Each node identifier maps to an equivalence class of the equivalence of sub- 
trees of the tree that it comes from. If there are only finitely many de Bruijn 
indices with non-trivial bounds, each of those is a regular binding tree, and the 
original subtype and supertype trees are regular binding trees, then the node 
identifiers map to a finite set of equivalence classes. So the pair of the current 



subtype and supertype node identifiers map to a finite set. Thus for any suffi- 
ciently long path for which there are initial or final subproblems there will be 
a repeat of this pair. This property is the key to showing completeness of the 
subtyping rules, it will be used to cut off the proof of sub typing to make a finite 
derivation, as we shall see. 

This characterisation of subtyping is also used to build an algorithm for decid- 
ing subtyping. Essentially, the algorithm constructs a deterministic finite-state 
automata that searches for paths for which the final subproblem fails, ff it fails 
to find such a path, that is, its language is empty, then the subtyping holds. It 
uses just the equivalence classes of the node identifiers as well as some informa- 
tion about which binder equivalence classes correspond — but this information is 
finite too if there are finitely many equivalence classes. The algorithm is able to 
do promotion, matching, and determining subtrees with just this information. If 
there are a finite number of equivalence classes (as will be the case for regular 
binding trees and finite number of non-trivial bounds) then the search space is 
finite and the algorithm is a decision procedure. 

3 Types 

Now I will define the syntactic type system, map it to trees, and then give a 
sound and complete set of type equality and subtyping rules. 

Let Var be a some set of type variables ranged over by metavariable a. The 
set of types, Type, ranged over by metavariables t and a is defined by this 
grammar: 

r ::= a | T | _L | t\— >T2 | Va < t\.ti | rec a.r 

subject to the requirement that in rec a.r, r is syntactically contractive in a, 
written r i a. The latter is defined by induction on the structure of t as follows: 

a' l a <= a ^ a' 

T I a 

_L I a 

t\ -^t 2 I a 

Va' < Ti .T2 4 a 

rec a'.r la <=a = a'\/T\.a 
3.1 Mapping Types to Regular Binding Trees 

Types map to trees given trees for the free variables. An environment, ranged 
over by metavariable r], maps type variables to trees, Env — Var — > Tree. An 
environment is distinguishing if it maps type variables injectively to {var(n) 
n G N} (note this is weaker than my previous work, which required bijectivity) . 
Shifting the free variables of an environment rj up and mapping a to the new 
free variable is shift(rj 7 a) and is straightforward to define. 



The meaning of a type in an environment is a tree denned by induction on 
the type as follows: 



treeof (a) v 
treeof (T)^ 
treeof ( IS) 71 



= ri{a) 
= T 
= _L 



treeof (ti^t 2 ) v = treeof '(ti) v -> treeof '(r 2 ) n 

treeof (Va < ri.r 2 ) r) = Vireeo/ {Ti) s Mft(r),a)- treeof \t 2 ) sUft^a.) 

treeof (rec a.r) v = fix(At. treeof (r)^ {a ^ t} ) 

where fix(/) is the unique fixed point of a contractive function / on trees (com- 
plete ultrametic spaces have unique fixed points for contractive functions) — the 
definition here is well defined as it is easy to prove that syntactic contractivity 
implies contractivity. 

The meaning of a type is a regular binding tree and any regular binding tree 
is the meaning of some type. 

Theorem 3. If i] maps type variables to regular binding trees then treeof (t) v is 
a regular binding tree. If t is a regular binding tree and rj can generate t 's free de 
Bruijn indices (there exists an a such that rj(a) = var(n) for each n such that 
t(p) = n + bind(t, e — > p)) then there is a type r such that treeof '{t) v = t. 



3.2 Equality and Subtyping Rules 

To motivate the subtyping rules, consider some particular problems. First, if a is 
bounded by T->a then should a be a subtype of rec a'.T— Va'l Intuitively, a is 
some set A of functions that take any value to a value in A] similarly rec a'.T— >a' 
is the set B of all functions that take any value to a value in B; it seems that A 
should be a subset of B, so the subtyping should hold. Using de Bruijn index 
for a then these types map to the trees var(O) and t 2 = {(R*, — >), (R*L, T)} with 
bound set /3 such that /3(0) = T^var(O). Let R = {(var(O), /3, t 2 ), (T, /3, T)}. 
Then R is a partial subtyping and so var(O) <p t 2 . If the subtyping rules arc to 
be complete then clearly they must be able to derive that a bounded by T^a 
is a subtype of rec a'.T— >a'. 

Second, consider an example that does not even involve recursive types. If 
a is bounded by (o^T)— >± then should a be a subtype of a— >T? These types 
map to trees var(O) and var(0)->T with (3 such that /3(0) = (var(O)— >T)— >±. 
Let R — {(var(O), /3, var(O)— >T), (_L, j3, T)}. Then R is a partial subtyping so 
var(O) <p var(0)^T and the subtyping for the types above should be derivable 
with the subtyping rules. 

Using the standard structural subtyping rules with the equality rules from 
Amadio and Cardelli to try to prove these subtypings results in a cycle — after 
some steps what needs to be proved is what we are trying to prove. Here is the 



attempt for the first subtyping (where B = a < T— >a and t 2 = rec af.T—tc/): 



I- T 2 = T— >T 2 

BhT<T 5ha<T 2 h T^t 2 = t 2 
BPTtoTT^ B h T^r 2 < r 2 
B h Q < T^a g h T— >q < r 2 

Bha<T 2 

Notice though that the steps make some progress, in that they use the structural 
subtyping rule for function types at least once, so coinductive proofs would prove 
this subtyping. A specialised rule for recursive types could prove this derivation, 
but in the other example, we really need something like coinduction (where 
B = a < (a^T)^_L): 



B\- a< q-^T flhl<T 

B h a < (a->T)-»± B h (a->T)-t± < a-TT 
B b a < a-^T 



I will present normal inductive rules that in the rules that make progress, namely 
the structural subtyping rules for function and forall quantified types, allow 
the conclusion to be assumed in proving the subterms to have the appropriate 
subtyping relationship. This modification of the standard rules is enough to get 
sound and complete rules with respect to the tree interpretation of types. 

Subtyping assumptions, ranged over by metavariable A, are sets of pairs of 
types, which I will write in the form n < <7i, . . . , r„ < <r n . Subtyping bounds, 
ranged over by metavariable B, have the form a\ < Ti, . . . , a n < r n where the a, 
are distinct. The meaning of subtyping bounds in a distinguishing environment 
is a bound set and is defined as: 

, . / treeof(n)r, ??(«*) = var(n) 

treeof [a\ < n, . . . , a n < t„)„ — An. < , , v ' \ 

J v ' [var(n) otherwise 

The rules for type equality and subtyping appear in Figure [5] and define two 
judgements, b ri = t 2 asserts that types n and t 2 are equal and ^4; i? h n < r 2 
asserts that Ti is a subtype of r 2 under assumptions A and bounds B. The equal- 
ity rules are those of Amadio and Cardelli. The interesting rules are eqroll and 
equnq. The former asserts that a recursive type is equal to its unrolling. The 
latter asserts that recursive types are unique — more specifically that two types 
that satisfy the same syntactically contractive equation are equal. It is key to 
proving completeness of the equality rules with respect to the tree interpreta- 
tion of types. The subtyping rules are also fairly standard. There are the usual 
reflexivity, transitivity, variable bound, top, and bottom rules. Rule STASSUME 
allows an assumption to be used. Rule STFUN is the usual structural subtyping 
rule except that the conclusion can be assumed while proving the the argument 
types are contravariantly related and the result types are covariantly related. 
Rule STALL is the Kernel rule for F-bounded forall quantified types, again where 



I- Tl = Tl 



\~ T2 = T\ h Tl = T2 h T2 = T3 

= i EQSYM = - - EQTRANS 

r Tl = T2 \~ Ti — T- A 



EQVAR — — — EQTOP • EQBOT 



h a = a hT = T h _L = _L 

H Tl = T'2 H (Tl = (T2 h Tl = T2 h (Tl = 0"2 

: - EQFUN = EQALL 

h Tl— >-(Tl = T2— S><72 h Va < Tl.CTl = Va < T2.CT2 

I- Tl = T2 



EQREC EQROLL 

: a.r} 

EQUNQ 



h rec a.n = rec a.T2 h rec a.r = r{a h-s> rec a.r} 

h Ti = a{a H> n} h T2 = <r{a H- T2} <t 4- a 



h Ti = t 2 



A-B\-T!<T2 



h Tl = T 2 v4; B h Tl < T 2 A;BhT 2 <T 3 

STREF — — STTRANS 



A;B\~ti<t 2 A;B\-ti<t 3 

ti < t 2 £ A a < r e B 

STASSUME , „ , STBOUND 



A; B h ri < r 2 A;fiha<r 

STTOP , - : STBOT 



A;B\-t<T A;Bh±<T 

A' = A,Va < Ti.cri < Va < T2.CT2 

A' = A, Tl— >(Jl < T2— >(T2 l~ Ti = T2 

A'; B h r 2 < Ti A'; B, a < r 2 h <7i < cr 2 

A'; B h ai < (72 a ^ /w(A) U MB) 

STFUN STALL 

A; B h Ti—^CTi < T2— >a2 A; B h Va < Ti.ci < Va < T2.<T2 



Fig. 2. Typing Rules 



the conclusion can be assumed when proving that the body types are covariantly 
related. 

We are mainly interested in judgements of the form 0; B h t\ < ra, which I 
will write simply as B h ti < T2 — the assumption sets are really just for internal 
use to prove such judgements. 

I proved the soundness and completeness of the equality rules in previous 
work [Gle02a:. I repeat those proofs for the system in this chapter in the com- 
panion technical report [Glel2j . and I will use them in proving the soundness 
and completeness of the subtyping rules. 



3.3 Soundness 



If two types are subtypes then the trees that they map to are subtypes. 

Theorem 4. If BY- T\< t^ and r\ is distinguishing then treeof (ti) v <treeof(B) 
treeof(T 2 )r,- 



Proof: The proof uses a generalisation of subtyping on trees that takes assump- 
tions into account, and is then by induction over the derivation of the subtyping 
judgement using a lemma that says that subtyping with assumptions has prop- 
erties similar to those of the rules. The soundness of the equality rules is also 
used for Rule stref. □ 

3.4 Completeness 

If the trees two types map to are subtypes then the rules can derive that they 
are subtypes. 

Theorem 5. If rj is distinguishing and treeof(Ti) n <treeof(B) treeoffo)^ then 
B\-n<T 2 . 

The proof is in the companion technical report |Glel2] . As previously men- 
tioned, the key to the proof is the characterisation of subtyping. It states that 
no final subproblem fails. Each final subproblem can be represented using node 
identifiers and these node identifiers come from a finite set of equivalence classes, 
thus on any sufficiently long path there will be a repeat of which equivalence 
class is the subtype and which equivalence class is the supertype. For any node 
identifier of the initial and final subproblems the proof builds a canonical type. 
Then the proof shows that Rule STBOUND can mimic promotion to a bound and 
that if two node identifiers match then one of the Rules eqvar and stref, ST- 
top, STBOT, STFUN, or stall can prove the required subtyping. For STALL the 
proof uses the fact that the left subtrees are equal and the completeness of the 
equality rules to show that the bound types are equal; for the right subtree and 
both subtrees of STFUN the proof recurses to a longer path, for which there are 
initial and final subproblems. Finally at a repeat in the equivalence classes the 
proof shows that the requried subtyping is in the assumption set and uses Rule 
STASSUME. Finally, in various places the proof needs to show that the canonical 
types match up to other types, which it does by showing that they generate the 
same trees and by using the completeness of the equality rules; Rule STTRANS 
is used to combine everything together. The proof is just going through all the 
details of the above sketch. 

4 Binding- Tree Automata 

This section defines a notion of tree automata that generate trees and a con- 
struction that determines subtyping — it takes two tree automata to a DFA whose 
language is empty exactly when the subtyping relation holds. 

A binding-tree automata is a quadruple (Q,i,<5, If) such that Q is a finite 
set of states, i G Q is the initial state, 5 : Q x {L, R} — 1 Q is the transition 
function, lf:Q-> {fv(ri) | n G N}U{bv(cj,f) | q G Q/\£ G {L, R}} U {T, _L, V} 
is the labelling function, S(q,£) is defined if and only if lf(q) G {— >,V}, and 
lf(q) = bv(q',l) only if If(q') = V and all paths from i to q go through q' and 
on the last time through q' they follow an t edge. Intuitively, a tree automata 



takes as input a path through a tree and outputs the node label at the end of 
that path, which can be either a free variable (of the original tree), a bound 
variable (that bound by the last time through the identified state), top, bottom, 
function, or forall. 

The tree that a tree automata generates is defined as follows: 

1 ql = V 
Oql^V 



bind(ql, £) 

shift(f,q:=n) = V- { + „ J J J 



5((q, n, f),£) = (5(q, t),n+ bind(lf(q), £), shift(f, q := bind(lf(q), £))) 

{n + m lf(q) = fv(m) 
f(q>) lf(q) = bv(q',£) 
lf(q) otherwise 
treeof(Q,i,8,lf) = Xp.lf(S*((i,0,Xq.O),p)) 

where S* is the obvious lifting of S to sequences of edges. The formal definition 
just tracks enough information to determine the de Bruijn indices for the states 
labelled as free and bound variables, otherwise it follows the intuition above. 

Binding-tree automata generate regular binding trees and all regular binding 
trees are generated by a binding-tree automata. 

Theorem 6. Ifta is a binding-tree automata then treeof(ta) is a regular binding 
tree. If t is a regular binding tree then there exists a binding-tree automata ta 
such that treeof(ta) = t. 



4.1 Subtyping Algorithm 

Now, I will define a construction that takes two binding-tree automata and 
produces a deterministic finite-state automata (in the usual sense), such that the 
DFA's language is empty if and only if the trees of the two binding-tree automata 
are in the subtyping relation. In particular, the DFA will search for paths that 
show that the two trees are not subtypes. The characterisation of subtyping tells 
us that such paths exists if and only if the trees are not subtypes. Most of the 
information for determining if states match after promotion is available from 
the labelling functions, but some additional information is needed. Specifically, 
the construction must track which binding states in one binding-tree automata 
correspond to which binding states in the other binding-tree automata, in order 
to determine if two bound variables match or not. This correspondence will be 
tracked by partial bijections, defined next. 

A partial bijection R between sets A and B is a set of pairs from A and 
B such that (ai,&i) G R and (02,62) £ R implies that a\ — a\ if and only if 
bi = b 2 . Partial bijection update is defined as: R{a ±=f b} = {(a' 7 b r ) e R \ a' ^ 
aA6'/6}U{(a, b)}. 

An automata bounds is a finite function from de Bruijn indices to binding- 
tree automata — de Bruijn indices without a bound are bounded by themselves. 



An automata bounds generates a bound set as follows: 



treeof(ba) = An. 



{ 



treeof (ba(n)) n £ dom(fea) 
var(n) n dom(fea) 



The input to the construction, an (automata) subtyping problem, is a triple 
(ta|_, ba, tax) where ta\_ and tap; are binding-tree automata and ba is an automata 
bounds. The construction will search over the various states of the various au- 
tomata, so define a problem state of (ta\_, ba, taR) to be either (L, q) for q a state 
of tat, (n, q) for n e dom(fea) and q a state of ba(n), or (R, q) for q a state of taR. 
Define the transition function, S, and the labelling function, If, for (ta\_,ba,taR.) 
by lifting the underlying transition functions and labelling functions in the ob- 
vious way. The states of the DFA are quadruples fa,<p,q 2 ,R) where q\ and q 2 
are problem states, (j> 6 {+,0,—} is a variance (+ means that qi should be a 
subtype of q 2 ; o means that q\ should be equal to 52; — means that q\ should 
be a supertype of q 2 ), and R is a partial bijection between problem states. 

I build the formal definition of the construction up in several pieces. First, I 
define how problem states are promoted, that is, if they are variables they are 
replaced with their bounds, as follows: q "~^(ta L .ba,taR) ( n >*) if Vfa = M n )i n e 
dom(foa), and i is the initial state of ba(n); and q ^ s t p S(q', L) if lf(q) = bv(q' , I). 

Second, a DFA state matches, matches s t p (qi,4>,q2,R), exactly when one of 
the following holds: 



- lf( qi ) = bv(^), lf(q 2 ) = bv(q' 2 ,£), and G R, 

- lf(qi) = T and </>=-, lf(q 2 ) = T and 4> = +, or lf{q{) = lf(q 2 ) = T, 

- lf(qi) = -L and </> = +, lf(q 2 ) = 1- and <j) = -, or lf(q x ) = lf(q 2 ) = _L, 

- lf(qi) = V fa) = or 

- iffa) = iffa) = y. 



Intuitively, a state matches if the base subtyping proposition holds for the nodes 
represented by that state. Using it and the notion of promotion, I can define 
a function that promotes a DFA state if possible to a matching DFA state. In 
particular, define promote stp (qi, +, q 2 , R) = (q[, +, q 2 , R) where q[ is the first q[ 
such that qi ^->* tp q[ and matches s t P (q'i, +, 92, R) or q[ = q\ if no such q[ exists; 
similarly, define promote 8tp (qi,—,q 2 ,R) = fa,—,q' 2 ,R) where q' 2 is the first q 2 
such that q 2 ^* stp q'2 an d matches stp (qi, — , q 2 , R) or q 2 = q 2 if no such q' 2 exists; 
define promote stp {q 1 ,o,q 2 ,R) = (q 1 ,o,q 2 ,R). 

Third, the subtree of a DFA state along an edge is defined as follows: 

subtree stp ((q 1 ,<t>,q 2 ,R),\-) = (Sfa, L), —(f), Sfa, L), R{qi ^ 172}) If fa) = -> 



subtree stp {fa,(p,q 2 ,R), L) = (Sfa, L),o,5(q 2 , L),R{qi <=> q 2 }) If fa) = V 
subtree 8tp (fa,4>,q 2 ,R), R) = (Sfa, R), 4>, Sfa, R),R{qi ^ q 2 }) If fa) E {->•, V} 



(Where — h = — , — o = o, and = +.) Intuitively it computes the state 

that corresponds to the left or right subproblem of a DFA state, updating the 
variance and binding correspondence in the appropriate way. 



Finally, the subtype automata is denned as follows: 

subtype(tai,ba,taR) = 
(QxVarxQx(Q^ Q), 
promote stp (q L ,+,q R ,9), 
X(q,£). promote stp (subtree stp (q, £)), 
{q | -^matches stp \q)}) 

where Q is the set of problem states of (ta\_, ba, tax), q\_ is the initial problem 
state of ta\_, and q R is the initial problem state of ta R . 

The DFA so constructed has an empty language exactly when the subtyping 
relation holds. 

Theorem 7. L{subtype{ta\_,ba,taR)) = if and only if treeof{ta\_) <t re eof(ba) 
treeof{ta,R) . 

Proof: (Sketch) First the proof shows that for all the paths for which there 
are initial subproblems that the states computed by the subtype DFA (in some 
sense) generate the corresponding trees of the subproblem. If the subtyping holds 
then all the final subproblems do not fail. The proof then shows that those paths 
match in the subtype DFA and are thus not in the language. The only other paths 
the DFA considers are following a left edge from a forall matching subproblem, 
where the DFA switches to invariance and becomes an equality checker. Since 
to forall match the left subtrees must be equal, none of those paths will be in 
the language. Conversely if the language is empty then the proof shows that 
the left edge from forall matching states implies the left subtrees are equal and 
thus the corresponding final subproblem does not fail, and that all other final 
subproblems do not fail from their paths not being in the language. Thus by 
characterisation of subtyping the subtyping holds. The rest of the proof is just 
working through all the tedious details. □ 
Since determining if the language of a DFA is linear time, the construction 
provides an exponential time algorithm for deciding subtyping (at least of au- 
tomata). 

4.2 Polynomial Time Algorithm 

The key to getting a polynomial time algorithm is that the binder correspondence 
information is only used in very limited ways in the subtype DFA, and so it 
can almost be ignored. For this section, fix a subtyping problem stp. Let Q 
be the problem states of stp, If the labelling function for problem states, and 
subtype(stp) — (Q' ,i,5' , F). 

A triple is binder correspondence independent, bci stp (qi, 4>, (72), exactly when 
<fi = + and lf{q2) is not a bound variable, <j) = an d either lf(qi) or lf{q2) is not 
a bound variable, or cf) = — and lf(qi) is not a bound variable; bci s t p (qi,4>, q 2 , R) 
exactly when bci stp (qi, <j>, 92)- If bci stp (qi,(t),q2) then matches stp (qi, <j), q 2 , Ri) if 
and only if matches s t p (qi, <fi, 92, R2) for any R\ and R 2 . Then observe that if 
5'*(i,p£) is defined then 5'*(i,p) is binder correspondence independent. Thus if 



5' (i,p) is binder correspondence independent then determining if p G F can be 
done without tracking the binder correspondence at all. 

Now consider p such that S'*(i,p) is binder correspondence dependent. By 
definition S'*(i,p) — promote stp (q\_, <f>, q R , R) for some qi, <fi, q R , and R. The first 
three can be determined without tracking the binder correspondence. Consider 
what information is needed to determine if p G F. Case 1, <f> = o; In this case 
lf(q L ) = bv(qj\4) and lf(q R ) = b\/(q' R ,£ R ) and p G F if and only (q' L ,q' R ) G R. 
Case 2, <f) = +: In this case £/(<?r) = hv(q' R ,£). If there is no q' L such that 
Wli i'r) e ^ then p G F. If there is then p G F if and only if q\_ ^* stp q' L . Case 3, 
(/>=—: similar to the previous case. 

My strategy for determining the above conditions is to compute facts of 
the form q\ q 2l q f, and f q at the triples that are binder correspondence 
dependent. The meaning of q\ ±=> q 2 is that there is an R possible at the triple 
with (</i, q 2 ) G R; similarly q f means there is an R possible at the triple with no 
q' such that (q, q') G R; and t q means there is an R possible at the triple with 
no q' such that (q' , q) G R. Computing such facts is a simple dataflow problem. 
If (qi, 4>, q 2 ) is such that lf(qi) = If (92) = V then q\ t=f q 2 is generated, q[ q' 2 
is propagated if % ^ q[, q t is propagated if g ^ gi, t 9 is propagated li q^ q 2 , 
qi <=; q where q ^ q 2 is changed to f q, and q ^ q 2 where q 7^ gi is changed to 
q j. Function states propagate all facts. 

In summary, the polynomial time algorithm computes the triples that are 
possible and checks that the binder correspondence independent ones match and 
sets aside the binder correspondence dependent ones. Then it sets up and solves 
the dataflow problem outlined above. Finally it uses the computed facts at the 
binder correspondence dependent triples to determine if they match or not. The 
algorithm is at worst 0(n 4 ) as quadratic dataflow facts need to be propagated to 
quadratic nodes, and the other phases are at least as good. It might be possible 
to do better by exploiting the scoping requirements of binders, but I have not 
explored this possibility. 



5 Discussion 

To put all the pieces together, all we need to do is define a way to go from 
types to binding-tree automata. In my previous work I defined exactly such a 
transformation — in particular given a type r and a distinguishing environment 
77 there is a binding-tree automata automataof „(t) constructable in linear time 
such that ireeo/(r) 7) = treeof (automataof „(t)). Combining that algorithm with 
the one in Section IH gives a polynomial-time algorithm for deciding subtyping 
on the types themselves. The soundness and completeness of the rules and cor- 
rectness of the algorithm means both that this algorithm is deciding subtyping 
according to the type rules, and that the type rules correspond to the tree in- 
terpretation of subtyping, which hopefully corresponds to our intuitive notion of 
what subtyping should be for the system under consideration. 



Theorem 8. Ifrj is distinguishing then: 

B h n < r 2 
<^> 

L(subtype(automataof n (Ti), automataof V (B) , automataof rj (r 2 ))) = 

This chapter considered the Kernel rule for subtyping forall quantified types. 
As previously mentioned, that rule is not the most general rule that is sound for 
such types. I believe that the definition of subtyping for binding trees and the 
typing rules can be modified for the most general rule and the soundness and 
completeness theorems can still be proven, but I have not done this. Nothing in 
the proofs is critically dependent on the bound being invariant. The construction 
of a DFA for subtyping, however, is critically dependent on the bound being in- 
variant. In particular, to augment the construction for the full rule first requires 
tracking which side is the tightest bound (easy to do), but also requires figur- 
ing out the binder correspondence after promoting to a bound, which requires 
saving the correspondence at the binding point leading to a linked list like cor- 
respondence information — no longer a finite set. Thus I believe this system is 
undecidable for similar reasons to full F< . 

There is another rule for subtyping forall quantified types that allows the 
bound to be contravariant but considers the variable unbounded in the body 
type. It is usually ignored as it leads to a lack of principal types. I believe 
that this rule could also be worked into my definition of trees, type rules, and 
automata construction — the variables bound by such quantifiers have no bounds, 
and so they act very similar to function types. I use a self quantifier in my object 
encoding that requires the body type to be covariant. As the body of the self 
quantifier is also the bound of the quantified variable, the typing rules cannot 
have an invariant bound. This variant with unbounded variables for checking 
the body is the most appropriate. The full rule is likely undecidable, and lack 
of principal types is avoided because the introduction form for self quantifiers 
includes a full type annotation, unlike for type lambdas that include only a 
partial type annotation. 

Thus I believe that the results of this paper can be extended to handle other 
base types, first-order constructs of various flavours, other bound rules, and 
other second-order constructs like existentials and self quantifiers. Extending to 
higher-kinded systems is definitely future work and might not be possible. The 
first problem is that F w with equirecursive types has the simply-typed lambda 
calculus with general recursion at the type level, hence guaranteeing termination 
is probably a necessary first requirement. 
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